Control system to automatically control vehicle headlamps

ABSTRACT

An automatic vehicle headlamp dimming system which includes an optical system and an imaging processing system. The optical system is configured to discriminate between headlamps and tail lamps, and focus the light rays from the headlamps and tail lamps on different portions of a pixel sensor array. The optical system as well as the image processing system provides for relatively increased discrimination of headlamps and tail lamps of other vehicles and also enables the high beam headlamps of the control vehicle to be controlled as a function of the distance as well as horizontal angular position of other vehicles relative to the controlled vehicle.

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is a continuation of U.S. patent applicationSer. No. 09/151,487, entitled “CONTROL SYSTEM TO AUTOMATICALLY DIMVEHICLE HEADLAMPS” filed on Sep. 11, 1998, by Joseph S. Stam et al.,which is a continuation of U.S. patent application Ser. No. 08/831,232,entitled “CONTROL SYSTEM TO AUTOMATICALLY DIM VEHICLE HEADLAMPS” filedon Apr. 2, 1997, by Joseph S. Stam et al., now U.S. Pat. No. 5,837,994.Priority under 35 U.S.C. §120 is hereby claimed on both the aboveapplications and the entire disclosures of each are incorporated hereinby reference.

BACKGROUND OF THE INVENTION

[0002] The present invention relates to a system for automaticallydimming vehicle high beam headlamps.

[0003] Regulations set forth by the United States Department ofTransportation (DOT) regulate the light emissions of vehicle high beamheadlamps. Various state regulations are used to control the amount ofglare experienced by drivers of other vehicles whether the vehicle istraveling in the same direction as the controlled vehicle or in anopposite direction.

[0004] Known vehicle high beam headlamp emissions in accordance with theDOT regulations provide an intensity of 40,000 cd at 0 degrees, 10,000cd at 3 degrees, 3250 cd at 6 degrees, 1500 cd at 9 degrees, and 750 cdat 12 degrees. An example of such an emission pattern is illustrated inFIG. 1. In order to avoid an illuminance of 0.5 foot candles (fc)incident on another vehicle, the vehicle high beam headlamps should bedimmed within 230 feet of another vehicle at 0 degrees, 115 feet ofanother vehicle at a horizontal position of 3 degrees relative to thedatum, and 65 feet in the position of the other vehicle is 6 degreesrelative to the controlled vehicle.

[0005] Various known head light dimmer control systems are known in theart. In order to prevent the drivers of other vehicles from beingsubjected to excessive glare levels, such automatic headlamp dimmersystems must sense both the head lights as well as the tail lights ofother vehicles. While many known systems are adequately able to detectheadlamps of oncoming vehicles, such systems are known to inadequatelysense tail lights of vehicles traveling ahead of the control vehicle. Assuch, such systems are not able to automatically dim the high beamheadlamps in time to prevent drivers of the vehicles travelling in thesame direction as the controlled vehicle being subjected to excessiveglare levels.

[0006] U.S. Pat. No. 5,537,003, assigned to the same assignee of thepresent invention, discloses an automatic headlamp dimming system whichincludes an optical system for sensing tail lamps as well as headlamps.The '003 patent discloses a single photo diode with a mechanicalscanning arrangement for scanning a predetermined field of view.Although the system provides relatively suitable sensing of headlamps aswell as tail lamps, the optical subsystem is rather complicated andexpensive to manufacture.

SUMMARY OF THE INVENTION

[0007] It is an object of the present invention to solve variousproblems in the prior art.

[0008] It is yet another object of the present invention to provide avehicle headlamp dimming system which eliminates the need for mechanicaloptical scanning systems.

[0009] It is yet another object of the present invention to provide aheadlamp dimming system that is adapted to dim the high beam head lightsat different distances as a function of the horizontal angular positionof another vehicle relative to the controlled vehicle.

[0010] Briefly, the present invention relates to an automatic vehicleheadlamp dimming system. The system includes an optical system and animaging processing system. The optical system is configured todiscriminate between headlamps and tail lamps and focus the light raysfrom the headlamps and tail lamps on different portions of a pixelsensor array. The optical system, as well as the image processingsystem, provides for relatively increased discrimination of headlampsand tail lamps of other vehicles and also enables the high beamheadlamps of the control vehicle to be controlled as a function of thedistance as well as the horizontal angular position of other vehiclesrelative to the controlled vehicle.

[0011] These and other features, advantages, and objects of the presentinvention will be further understood and appreciated by those skilled inthe art by reference to the following specification, claims, andappended drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0012] These and other objects of the present invention will be readilyunderstood with reference to the following specification and attacheddrawing, wherein:

[0013]FIG. 1 is a top view illustrating the headlamp emission pattern ofa conventional high beam headlamp.

[0014]FIG. 2 is a side cross-sectional view of the optical system, whichforms a part of the present invention illustrating light rays incidentat a vertical angle within the desired field of view.

[0015]FIG. 3 is similar to FIG. 2 illustrating the light rays incidentat a vertical elevation angle beyond the desired field of view.

[0016]FIG. 4 is a top cross sectional view of the optical systemillustrated in FIG. 1 illustrating the light rays at a horizontal anglewithin the desired field of view.

[0017]FIG. 5 is a block diagram of the automatic head light dimmingsystem in accordance with the present invention.

[0018]FIG. 6 is an overall flow diagram of the image processing inaccordance with the present invention.

[0019]FIG. 7 is a flow diagram illustrating the method for detectingtail lamps of vehicles within the desired field of view.

[0020]FIG. 8 is a flow diagram for detecting headlamps from othervehicles within the desired field of view.

DETAILED DESCRIPTION OF THE INVENTION

[0021] The automatic headlamp dimming system in accordance with thepresent invention includes an optical system as illustrated in FIGS. 2-4and an image processing system as illustrated in FIGS. 5-8. In order toenable the high beam headlamps to remain on for the longest reasonabletime without subjecting the driver of another vehicle to excessiveglare, the automatic headlamp dimming system in accordance with thepresent invention controls the vehicle high beam headlamps as a functionof the distance as well as the horizontal angular position of the othervehicle relative to the controlled vehicle. As will be discussed in moredetail below, the optical system is adapted to discriminate betweenheadlamps and tail lamps of other vehicles. The light rays from theheadlamps and tail lamps of other vehicles are spatially segregated on apixel sensor array to provide increased discrimination of headlamps andtail lamps relative to other ambient light sources, such as road signsand reflections from snow and the like. The optical system enables boththe horizontal and vertical position of incident lights sources to bedetermined within the field of view of the optical system. The imageprocessing system processes the pixels to provide for automatic controlof the headlamps as a function of the distance and horizontal angularposition of another vehicle relative to the control vehicle. As such,the system in accordance with the present invention is adapted toprovide optimal control of the vehicle high beam headlamps by allowingthe high beam headlamps to remain on for as long as possible whilepreventing the driver of the other vehicle from being subjected to anundue amount of glare.

OPTICAL SYSTEM

[0022] Referring to FIGS. 2-4, the optical system includes a pair oflenses 103 and 104, a lens holder 105, and an image array sensor 106. Asbest shown in FIGS. 2 and 3, the lenses 103 and 104 are verticallyspaced apart in order to allow imaging of substantially the same fieldof view onto different portions of the same array. The lenses 103, 104image generally the same fields of view because the distance between thelenses 103, 104 is relatively small relative to the light sources withinthe field of view of the device.

[0023] The lens 103 may be formed with a red filter dye for transmittinglight with wavelengths greater than 600 nm and focusing red light rays101 from tail lamps onto one-half of the image array sensor 106. The redfilter dye causes the lens 103 to absorb all light rays at the blue endof the visible spectrum and transmit light rays at the red end of thespectrum. As such, the amount of light transmitted from non-red lightsources, such as headlamps, is greatly reduced while light rays fromtail lamps are fully transmitted through the lens 103. As such, therelative brightness of the light rays from tail lamps imaged onto theimage array sensor 106 is greatly increased.

[0024] The lens 104 may be formed with a cyan-filtered dye fortransmitting light with wavelengths less than 600 nm. The lens 104 isused to focus the light rays onto the other half of the image arraysensor 106. The cyan filter dye has a complementary effect to the redfilter described above. In particular, the red filter dye absorbs lightfrom the blue end of the visible spectrum while transmitting light fromthe red end of the spectrum. As such, most of the light from sources,such as head lights, is transmitted through the lens 104 while virtuallyall of the light emanating from tail lamps is blocked.

[0025] Both headlamps and tail lamps emit a substantial amount ofinfrared light. By utilizing lenses with a filter dye or separatefilters which inhibit light at wavelengths greater about 750 nm, theinfrared light transmitted by the headlamps and tail lamps will besubstantially blocked by the lenses 103 and 104. By eliminating infraredlight, the ratio between intensity between red lights imaged through thered filter and red light imaged through the cyan filter will besubstantially increased.

[0026] The use of the red and cyan dyes for the lenses 103 and 104 ismerely exemplary. The filter characteristics of the lenses 103 and 104are selected to optimize the sensitivity of the device to specific lightsources. For example, the headlamps or tail lamps in new vehicles may bereplaced with alternative light sources with different spectralcomposition, for example, with high intensity discharge headlamps andlight emitting diode tail lamps requiring different filtercharacteristics. Depending on the spectral characteristics of theheadlamps and tail lamps, transparent lenses 103 and 104 may be utilizedwith separate color filters.

[0027] The lenses 103 and 104 may be formed as acrylic spherical lenses.Alternatively, the lenses 103 and 104 may be formed as aspherical lensesin order to minimize color dispersion and spherical aberration presentwith spherical lenses. Complex lenses formed from both spherical andaspherical lenses are also contemplated.

[0028] A single lens may also be used in place of the separate lenses103 and 104. The use of a single lens may be used to image the field ofview onto a full or partial color image array sensor containingpigmentation on the individual pixels in the array.

[0029] As shown best in FIGS. 2 and 3, the horizontal distance betweenthe two lenses 103 and 104 and the image array sensor 106 is slightlydifferent. Offsetting of the two lenses 103 and 104 compensates for thecolor dispersion created as a result of the fact that the index ofrefraction of materials varies with the wavelength of light transmittedthrough it. Because the two lenses 103 and 104 transmit differentportions of the visible spectrum, the distance between the lenses 103and 104 and the image array sensor 106 is optimized to minimize thedispersion for the band of light transmitted by each of the lenses 103and 104.

[0030] As mentioned above, the light rays 101 transmitted through thelens 103 are imaged onto one-half of the image array sensor 106 whilethe light rays 102 transmitted through the lens 104 are imaged onto theother half of the image array sensor 106. In order to provide suchspatial segregation of the light rays transmitted through the lenses 103and 104, the lens holder 105 is provided with cutouts 107 and preferablyformed or coated with a light absorbing material. These cutouts 107prevent light rays beyond the desired maximum vertical angle transmittedthrough the red lens 103 from being imaged onto the portion of the imagearray sensor 106 reserved for the light rays 102. Conversely, thecutouts 107 also prevent light rays transmitted through the lens 104from being imaged onto the portion of the image array sensor 106reserved for the light rays 101.

[0031] The field of view of the optical system is defined by theconfiguration of the lenses 103 and 104 and the cutouts 107 relative tothe image array sensor 106. For example, an exemplary field of view of10 degrees in the vertical direction and 20 degrees in the horizontaldirections may be created by the configuration set forth below. Inparticular, for such a field of view, the lenses 103 and 104 areselected with a diameter of 1.5 mm with a small portion cut away toallow the lenses 103, 104 to be positioned such that their centers areseparated by 1.0 mm. The lens 103 is positioned 4.15 mm from the imagearray sensor 106 while the lens 104 is positioned 4.05 mm away. Both thefront and rear surface radii of the lenses 103 and 104 are 4.3millimeters with a 0.2 millimeter thickness.

[0032] As best shown in FIGS. 3 and 4, circular cutouts 108 are formedin the lens holder 105. A pair of generally rectangular apertures 110 isformed in a rear wall 112. The rear apertures 110 are 1.6 millimeters inthe horizontal direction and 0.8 millimeters in the vertical direction.As best shown in FIG. 4, the cutouts 107 taper from the rear apertures110 to the diameter of the front cutouts 108 to provide the field ofview discussed above.

[0033] The configuration described above is thus able to baffle lightoutside of the desired horizontal and vertical field of view. Inparticular, FIG. 3 illustrates how the system baffles light raysincident at angles beyond the vertical field of view. FIG. 4 illustrateslight rays being imaged onto the image array sensor 106 within thehorizontal field of view.

[0034] The image array sensor 106 may be CMOS active pixel image sensorarray, for example, as disclosed in U.S. Pat. No. 5,471,515, herebyincorporated by reference and available from Photobit LLC of LaCrasenta, Calif. CMOS active pixel image sensors provide relatively highsensitivity and low power consumption as well as the ability tointegrate other CMOS electronics on the same chip. The image arraysensor 106 may be a 50×50 40 μm pixel array. The number of pixels in theimage array sensor 106 is selected such that not all pixels fall withinthe area that the lenses 103 and 104 project onto. The extra pixelsallow for simple correction for mechanical misalignment by offsettingthe expected image location.

[0035] The image array sensor 106 provides spatial information regardinglight sources within the field of view. The number of pixels present inthe array is selected to obtain sufficient spatial detail although thesize of the array is not limited and may be selected, and may even bedictated by physical and economic limitations.

[0036] The image array sensor 106 must be sensitive to accurately detecttail lights at several hundred feet. Such sensitivity may be achieved bylengthening the amount of time the photosites in the array are exposedto light during a frame period. A frame period is selected to enable thearray to capture and transfer a frame to the image processing system ina short enough time to allow the image processing system to detectanother vehicle entering the field of view. A short time period alsolimits the amount of motion within a frame during the integration periodand thus produces a relatively more accurate instantaneous image.

[0037] The use of a pixel array also provides other benefits. Forexample, as mentioned above, the light integration time to capture aframe can be varied. Such a feature allows the system to provide optimalresults in varying degrees in darkness. Another important aspect of animage array sensor is the ability to utilize subsets of the pixelswithin the array or an individual pixel. As such, as the window size isdecreased, the readout rates can be increased. Such a feature allows thesystem to discriminate ambient light sources, such as street lamps. Inparticular, such a feature allows the system to locate a light sourcewithin the frame and capture several samples of the light sources at arate several times greater than 60 Hz. In particular, if the samplesexhibit 120 Hz intensity modulation, the light source is likely a streetlamp or other light source powered from a 60 Hz AC power supply. If thelight source is not modulated, the light source is likely powered by thevehicle's DC power supply.

[0038] Another potential benefit of the image array sensor is that itallows the field of view immediately in front of the vehicle to beimaged by a higher pixel resolution. Thus, the system may be configuredsuch that the effective pixel resolution decreases as the angle of thevehicle relative to the control vehicle increases thus reducing theamount of processing time in those areas. Such a configuration reducesthe sensitivity of the device to light sources from reflectivestationary objects on the side of the road.

[0039] An image array sensor could be manufactured in which the pixelpitch is varied as a function of the area in the field of view that thepixel images. For example, pixels imaging the space corresponding tohorizontal angles within 3 degrees of the center of the vehicle may beprovided with a 10 μm pixel pitch. Pixels imaging horizontal anglesbetween 3 and 6 degrees may be provided with a 20 μm pixel pitch, whilethose imaging angles greater than 60 degrees may be provided with a 40μm pitch. While such a configuration may not increase the sensing area,the ability to resolve detail increases; an important aspect inconsidering the relative size of a tail lamp at a relatively largedistance. For example, a 4½ inch diameter tail light at a distance of200 feet subtends an angle of less than 0.11 degrees. If a 50×50 imagearray sensor is used to image a 20 degree field of view, the tail lampwould subtend approximately 8% of the total area imaged by the pixel.

[0040] A tail lamp is relatively brighter than its ambient surroundings;however, the red light contributed by the tail light is diluted by theambient light at such a distance. Such a factor is critical whencomparing the amount of red light in a given area to the amount ofnon-red light in the same area. When the area of space compared is largerelative to the light source, the percentage of red light is diminished.By comparison, if 10 μm pixels are used in the center of the array 106instead of 40 μm pixels, the tail lamp would subtend 0.04% of the totalareas, an improvement of 16 times.

IMAGE PROCESSING SYSTEM

[0041] The image processing system is illustrated in FIGS. 5-8. Theimage processing system includes the image array sensor 106; amicroprocessor 204, for example, a Motorola type MC68HC08XL36; aheadlamp control unit 205; and a pair of headlamps 206. As mentionedabove, an active pixel array sensor may be utilized for the image arraysensor 106. Such an active pixel array sensor includes an image array201 and an analog to digital converter (ADC) 202. A timing and controlcircuit is used to control the image array 201 as well as the ADC 202 tocontrol the integration time, read out timing, pixel selection, gainoffset and other variables. The microprocessor 204 is used to analyzethe data collected by the image array sensor 201. The microprocessor 204is in communication with the headlamp control unit, a conventional unit,implemented, for example, as a relay, which, in turn, controls theheadlamps 206. The headlamp control unit 205, in turn, changes thevoltage applied to the headlamp 206 to cause the high beam or brightlamp to be switched on or off.

[0042] The flow chart for the headlamp control is illustrated in FIG. 6.The system runs in a continuous cycle with occasional interrupts forabsolute light measurements, adjustments of ADC parameters, or otherfunctions.

[0043] At the beginning of each cycle, two images are acquired throughthe lenses 103 and 104. In step 302, the images from the lenses 103 and104 are analyzed to detect tail lamps. Another image is acquired in step303 through the lens 104. The image acquired through the lens 104 isacquired with a low enough gain to detect oncoming head lights whilerejecting lower light level reflections and nuisance light sources.After the images are analyzed, the system checks for very bright lightsin the image indicating the sudden appearance of vehicle headlamps ortail lamps within the field of view, as is the case when a car turns infront of the controlled vehicle in step 305. If bright lights aresensed, the device dims the headlamps 206 immediately and bypasses thetime verification as discussed below. The cycle is then repeated. Ifthere were no bright lights, the system proceeds to step 307 todetermine if there are any headlamps or tail lamps in the image.

[0044] In order to confirm the presence or lack of presence of aheadlamp or tail lamp in a frame, an undim counter and a dim counter areused. These counters provide verification of a particular light sourcewhether from a tail lamp or headlamp from consecutive frames beforesignaling the headlamp control unit 205 to dim or undim the headlamps206, except as described above, when a bright light is detected. Byproviding verification, anomalies within the device or in the image willnot cause spurious operation of the headlamps 206.

[0045] The dim counter is incremented each time a frame with a headlampor tail lamp is detected until the number of required consecutive framesto take action are reached. The dim counter is set to 0 each time aclear frame is processed. The undim counter is incremented with eachclear frame and set to 0 with each frame containing a headlamp or taillamp. The actual number of consecutive frames required to dim or undimis determined by the overall speed of the device. The more frames usedfor verification, the less susceptible the system will be to noise andanomalies. However, the device must be able to react quickly to beeffective so the number of verification frames is kept relatively low.Whenever a headlamp or tail lamp is detected in step 307, the undimcounter is set to 0 in step 308. In step 309, the system checks whetherthe headlamp 206 high beams are on. If the high beams are off, nofurther action is required and the cycle is repeated as indicated bystep 317. If the high beams are on, the dim counter is incremented instep 310. After the dim counter is incremented in step 310, the systemchecks in step 311 if the dim counter has reached the number ofconsecutive frames required to dim the headlamps 206. If so, the systemproceeds to step 306 and dims the headlamps 206 and resets both the dimand undim counters and repeats the cycle. Otherwise, the system repeatsthe cycle and proceeds to step to 317.

[0046] In step 307, if there are no headlamps or tail lamps in theimage, the dim counter is set to 0 in step 312. Subsequently, in step313, the system determines whether the high beams 206 are on. If thehigh beams are on, the system exits repeats the cycle in step 317. Instep 313 if the brights are not on, the undim counter is incremented.After the undim counter is incremented, the system checks in step 315whether the undim counter has reached the number of consecutive clearframes required to activate the high beams 206. If so, the high beamsare turned on in step 316, and the cycle is repeated. If the undimcounter is less than the required number for activating the brightheadlamps 206, the system repeats the cycle in step 317.

[0047] The flow diagram for tail light processing is illustrated in FIG.7. As will be discussed in more detail below, the primary method ofidentifying an object such as a tail light involves comparing the grayscale value of a pixel through the lens 103 to a gray scale value of thepixel representing the same space imaged through the lens, 104. If thevalue of the pixel imaged through the lens 103 is significantly higherthan the value of the pixel imaged through the lens 104, the lightsource is determined to be red light. In addition to determining if thelight is red, the system also checks the brightness of the red lightbefore deciding that the light is a tail lamp by determining if the grayscale value of the pixel is greater than a threshold value. As is knownin the art, the brightness of a light source varies with the square ofthe distance of the light source from the observer. As such, anapproximate determination of the distance of a leading vehicle can bemade to determine the appropriate time to dim the headlamps.

[0048] The threshold value may be computed in a variety of ways. Forexample, it can be a predetermined fixed number or a number that is afunction of the current image sensor and ADC settings. The thresholdvalue can also be determined by computing a threshold as a factor of theaverage pixel intensity of the entire image which would help eliminatevariances caused by changing ambient light sources. In addition, thepixel value may be compared to the average of the pixels in theimmediate area of the pixel of interest. This local average methodprevents relatively large, moderately bright spots in the image frombeing seen as vehicle light sources. More particularly, distant taillamps subtend less than one pixel and thus will only have moderatebrightness. Large spots in the image with moderate brightness are mostlikely caused by reflections from large objects. Close tail lamps whichsubtend many pixels will have a saturated center which will be brighterthan the surrounding pixels allowing the same method to detect them aswell.

[0049] The threshold may also be determined by varying the thresholdspatially by way of a lookup table or computation. However, thethreshold should be determined so that dimming occurs appropriately forthe dimmest tail lights allowed by the DOT standards. Distant vehiclesare subjected to the most intense portion of the controlled vehicle highbeam, thus requiring dimming only directly in front of the controlledvehicle as indicated in FIG. 1. Thus, a relatively low threshold may beselected for light sources imaged directly in front of the controlvehicle while a higher threshold for light sources that are not directlyin front of the control vehicle. For example, as discussed in connectionwith FIG. 1, the threshold for pixels imaging the field of view 3degrees right and left of the center should correspond to a light levelincident on the image array sensor 106 about 4 times as bright as thethreshold for red light directly in front of the vehicle and 12 times asbright for vehicles at 6 degrees. Such a spatially varying thresholdhelps eliminate false tail lamp detection caused by red reflectors bymaking the system less sensitive to areas of the sides of the controlvehicle.

[0050] A similar approach can be taken for varying the threshold forpixels in imaging areas of space and angles above and below the center.However, a more conservative approach can be taken when determining thetail light sensitivity relative to the vertical angle since vehiclestend to move more frequently and rapidly in vertical directions due tohills and bumps in the road. Therefore, by specifying relatively tightvertical thresholds may cause the bright headlamps 206 to switch on andoff as the vehicle moves several degrees up and down.

[0051] A hysteresis multiplier may be applied to the threshold toprevent oscillations of the headlamps 206 when the light source has agray scale value at or near the threshold. Thus, if the bright headlamps206 are off, the threshold will be lower for all pixels to prevent thebright headlamps from coming back on, even if the faintest tail lampsare present in the image. However, if the bright headlamps 206 are on,the threshold should be higher so that only tail lamps of sufficientbrightness are sensed to indicate that the car is within the dimmingrange to cause the headlamps 206 to dim.

[0052] One of the biggest problems facing the detection of the taillamps is the nuisance red light reflected from corner cube reflectorscommonly found as markers on the side of the road and on mailboxes. Thevariable threshold method mentioned above helps eliminate some of thisnoise. However, when a vehicle approaches a reflector at the properangles, it is relatively impossible to distinguish a red reflector froma tail lamp. Fortunately, by examining successive frames andinvestigating the motion of these objects over time, such reflectionscan be filtered. By storing the location of the tail lamps and imagesover time or by sensing small regions of interest where the tail lamp islocated several consecutive times, the device can look for rightwardmotion and determine if the light source is a reflector. Additionally,the speed at which the control vehicle overtakes a stationary object ismuch greater than the relative rate a vehicle would overtake anothermoving vehicle. Thus, the rate of increase in brightness of the objectwould be typically much greater for a stationary reflector than foranother vehicle. This rate of change in brightness coupled withrightward horizontal motion can be used as signatures to reduce thenumber of false tail lamps detected.

[0053] A computationally simpler method of analyzing spatial motion of alight source is to simply take several consecutive regions of the localregion of interest where the light source is located. Motion in thevertical and horizontal directions is relatively slow for tail lamps ofa leading vehicle. Simply sampling a pixel a few consecutive times tosee if the tail lamp is present in all samples can adequately eliminateobjects which rapidly move within the image.

[0054] The flow diagram for tail lamp processing is illustrated in FIG.7. Initially, in step 318, the system ascertains if the pixel is withinthe tail lamp window. In particular, as mentioned above, red lights areimaged onto half of the image array sensor 106. Thus, if the pixel isnot within the appropriate half of the image array sensor 106, thesystem proceeds to step 319 and moves to another pixel. As mentionedabove, there are two criteria for ascertaining whether the image is atail lamp. The first criteria relates to comparing the gray scale valueof the pixel image through the lens 103 with a corresponding gray scalevalue for the same area in space imaged through the lens 104. If thegray scale value of the pixel imaged through the lens 103 issignificantly larger than the gray scale value of the correspondingpixel imaged through the lens 104, one of the criteria for detecting atail lamp is met. Thus, if the pixel of interest is within the tail lampwindow as ascertained in step 318, the gray scale value of the pixelimaged through the lens 103 is compared with the gray scale value of acorresponding pixel imaged through the lens 104 in step 320. If the grayscale value of the pixel image through the lens 103 is not n% greaterthan the corresponding pixel imaged by the lens 104, the system proceedsto step 319 and examines another pixel. Otherwise, the system proceedsto step 321 and calculates the threshold for the particular pixel basedon the region of space it images. For example, as discussed above, thepixel thresholds may be varied based on their spatial relationshipwithin the image array sensor.

[0055] As discussed above, the other criteria for tail lamp detectionrelates to the relative brightness of the pixel relative to neighboringpixels. Thus, in step 322, the system calculates the average gray scalevalue of neighboring pixels. If it is determined in step 323 that thepixel gray scale value for the pixel imaged through the lens 103 is n%greater than the average gray scale value of the neighboring pixels, thesystem proceeds to step 324 and adds the pixel to the tail lamp list forfuture frames of reference. Otherwise, the system moves to step 319 andmoves the next pixel. In steps 325 and 326, the systems determineswhether or not the red light detected is a tail lamp or a reflector, asdiscussed above. If it is determined that the light is a reflector, thesystem proceeds to step 327 and moves on to the next pixel. Otherwise,the headlamps are dimmed in step 328.

[0056] The flow diagram for head light processing is illustrated in FIG.8. Headlamp detection is similar to tail lamp detection. The primarydifference is that only the lens 104 is utilized. As mentioned above,the pixel integration time is shorter and the ADC parameters are suchthat the image only shows very bright objects, such as headlamps. Mostreflections have low intensity light sources which fall well below thezero threshold of the ADC. As such, pixels are compared to the localaverage intensity of the neighboring pixels. Spatial variances in thethresholds may be set so that pixels corresponding to the center of thefield of view are more sensitive and pixels to the left of the image(left hand drive countries) have higher thresholds. These thresholds,however, should not vary spatially to the same degree as the thresholdfor the tail lamps because of the relatively wide variance in theemission patterns observed from headlamps. In addition, due to therelatively higher potential for more glare to the driver of an oncomingcar, the headlamps may be controlled and dimmed relatively more rapidlythan in the case when a tail lamp from a vehicle traveling in the samedirection is detected. Similar to the tail lamp processing circuithysteresis may be added to prevent cycling of the headlamps.

[0057] An additional concern with headlamp detection arises from therapid decrease in distance between oncoming vehicles which becomesespecially critical when an oncoming vehicle suddenly enters thecontrolled vehicle's field of view, for example, when turning a corneror in a similar situation. For this reason, an additional flag is usedto cause the vehicle to immediately dim the bright headlamps and bypassany verification if the light source is above certain absolutehigh-level brightness thresholds.

[0058] The primary nuisance light source complicating headlamp detectioncomes from overhead lights, such as street lights and electricallyilluminated street signs. One method of eliminating such nuisance lightsources is to analyze their motion. In particular, all overhead streetlamps will move vertically upwards in the image as the controlledvehicle is moving. Analyzing this motion provides an efficient method ofdetecting some street lamps. Unfortunately, distant street lamps appearto be at almost the same elevational angles as distant head lights andthe rate of vertical climb in the image does not become great until thestreet lamp is closer. However, as discussed above, street lighting isAC controlled and thus is subject to 120 Hz intensity modulation.Headlamps powered by DC source do not exhibit this characteristic. Thus,the image array sensor 106 is able to utilize a small number of pixelsfor taking several rapid consecutive readings in a window. If the windowis small enough, the window can read several hundred frames per second.Once the light source is identified in the image, several frames areread out at a rate of 240 Hz or higher. These readings are then analyzedto detect the intensity modulation. If modulation is present, the lightsource originates from an AC source and can be ignored. Alternatively, aphotodiode can be used in conjunction with a low pass filter todetermine the ratio of light in the image that was AC modulated to theunmodulated light. If a significant portion of the light source is ACmodulated, the light source present in the image is assumed to be fromAC light. Otherwise, the light source is assumed to be from a DC source.

[0059] The flow diagram for headlamp processing is illustrated in FIG.8. Initially, the system determines in step 329 whether the pixel is inthe headlamp window (i.e., in that portion of the image array sensor 106reserved for light arrays imaged through the lens 104). If not, thesystem proceeds to step 330 and examines the next pixel. Otherwise, thesystem examines the pixel in step 331 to determine if the pixel ismodulated at 120 Hz as discussed above. If so, the light source isassumed to be a street lamp and thus, the system proceeds to the nextpixel in step 330. If the pixel is not subject to 120 Hz intensitymodulation, the system then computes the average gray scale ofneighboring pixels in step 332. In step 333, the system determines thethreshold for the particular pixel based on the area of the space itimages. The system next compares the gray scale value of the pixel withan absolute high level threshold in step 334, for example, to determineif any oncoming cars suddenly come into the field of view of thecontrolled vehicle. If so, the system proceeds to step 335 and sets aflag to cause immediate dimming. Otherwise, the system proceeds to step336 and determines if the gray scale value of the pixel is n% greaterthan the average of neighboring pixels. If not, the system returns tostep 330 and examines the next pixel. Otherwise, the system proceeds tostep 337 and adds the pixel to the headlamp list for future frames toreference.

[0060] As discussed above, the system examines light sources asdiscussed above in steps 338 and 339 to determine if the light source isa street lamp. If the system determines that the light source is not astreet lamp, the system proceeds to step 340 and sets a flag to causedimming of the headlamps 206. If the system determines that the lightsource is a street lamp, the system proceeds to step 341 and moves on tothe next pixel.

[0061] Traditional vehicle lamps systems have the option of the brightlamps being either on or off. The present invention is readily adaptablefor use with a headlamp system where the brights can be activated tovary the brightness based on the distance of other vehicles in the fieldof view. In such an embodiment, the brightness of the headlamps may bevaried by various techniques including modulating the duty cycle of theheadlamp in order to reduce or increase the overall brightness level.

[0062] Variable intensity headlamps also result in better noisefiltration. In particular, whenever a light source is detected whichcauses the brightness of the controlled headlamps of the vehicles to bedecreased, other images can be detected to determine if the intensity ofthese other light sources decreases by a similar amount. If so, thesystem would be able to determine that the light source is a reflectionfrom the vehicle's headlamps. Such information can be used as feedbackto provide a relatively efficient means for eliminating nuisance lightcaused by reflections of the control vehicle headlamps. In such anembodiment, the brightness threshold discussed above would not be used.More particularly, the brightness of the brightest headlamp and taillamp in the images is used to determine the brightness of the controlledvehicle's headlamps. The brighter the headlamps or tail lamp in theimages, the dimmer the controlled headlamps.

[0063] Obviously, many modifications and variations of the presentinvention are possible in light of the above teachings. Thus, it is tobe understood that, within the scope of the appended claims, theinvention may be practiced otherwise than as specifically describedabove.

[0064] The above description is considered that of the preferredembodiments only. Modifications of the invention will occur to thoseskilled in the art and to those who make or use the invention.Therefore, it is understood that the embodiments shown in the drawingsand described above are merely for illustrative purposes and notintended to limit the scope of the invention, which is defined by thefollowing claims as interpreted according to the principles of patentlaw, including the doctrine of equivalents.

The invention claimed is:
 1. A control system to control the headlampsof a vehicle comprising: an image array sensor; and an optical systemconfigured to image the scene forward of the controlled vehicle ontosaid image array sensor, said optical system including two or morelenses each configured to image objects emitting light within apredetermined spectral band onto different portions of said image arraysensor, wherein the optical geometry of each lens is configured as afunction of the predetermined spectral band within which the lens is totransmit light.